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Abstract. Let T : H —> RU {+ 00 } be a closed convex proper function on a real Hilbert space H, and 
9$ : W H its subdifferential. For any control function e : R_|_ —^ R_)_ which tends to zero as t goes to 
+ 00 , and A a positive parameter, we study the asymptotic behavior of the trajectories of the regularized 
Newton dynamical system 

v ( t ) £ 9$ (x (t)) 

\x (t) + v (t) + v (f) + e (t) x ( t) = 0 . 

Assuming that e (t) tends to zero moderately as t goes to + 00 , we show that the term e (•) x (•) asymptotically 
acts as a Tikhonov regularization, which forces the trajectories to converge to a particular equilibrium. 
Precisely, when C = argmiriT 7 ^ 0, and e(-) is a “slow” control, i.e., /q^°° £ (t) dt = + 00 , then each trajectory 
of the system converges weakly, as t goes to + 00 , to the element of minimal norm of the closed convex set C. 
When <f> is a convex differentiable function whose gradient is Lipschitz continuous, we show that the strong 
convergence property is satisfied. Then we examine the effect of other types of regularizing methods. 


1. Introduction 

Throughout this paper, is a real Hilbert space with scalar product (•,•), and ||cc|| 2 = ( x,x ) for any 
x £ 1~L. Given <f> : T-L —> RU {+ 00 } a closed convex proper function, we will analyze some asymptotic 
viscosity selection properties for the regularized Newton dynamic governed by <h. 

Let us first recall some basic facts about this dynamical system. Given A a positive constant, the Regu¬ 
larized Newton dynamic ((RN) for short) attached to solving the minimization problem 

(V) min $ ( x ) 

xG'H 

is written as follows 

( 1 ) v ( t) e 9$ (x ( t )) 

(2) \x ( t) + v (t) + v ( t) = 0, 
where the subdifferential of $ at 2 ; £ dom$ is classically defined by 

<9<h ( 2 :) = {p £ T~L : $ ( y) > $ (x) + (p, y — x) Vy £ H} . 

When $ is a smooth function, (RN) is equivalent to 

Ax (t) + V 2 $(x(t)) + V$( 2 :{t)) = 0 

where A acts as a Levenberg-Marquard regularization parameter of the continuous Newton equation, whence 
the terminology. This dynamical system has been first introduced in 0, 0. Its extension to the case of two 
potentials gives rise to a new class of forward-backward algorithms, see m.0, m- in®, for a general closed 
convex and proper function <f>, it is shown that the Cauchy problem for the (x,v) system (fT ]l -(|2 |) admits a 
unique strong global solution. In addition, under the sole assumption that C = argmin$ ^ 0, for any orbit 
of Q-©, x ( t) converges weakly to an element of C, as t goes to + 00 . 

In many applications, a particular stationary solution is more interesting than others due to physical, 
economic or design considerations. When we have the global convergence of trajectories, one could let the 
trajectory reach a particular target equilibrium by appropriately adjusting the initial conditions. Never¬ 
theless, in many practical situations it is not possible to have an accurate control of the initial state. An 
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alternative approach consists in introducing a term into the system which forces convergence to the desired 
stationary solution, independently of the initial state. Such a term should vanish at infinity in order to 
recover, at least asymptotically, an equilibrium point of (U)-©. 

The above discussion motivates the introduction of the following abstract evolution system: 

(3a) v ( t ) G (x (t)) 

(3b) Ax (t) + v (t) + v (t) + £ (t) x (t) = 0, 


where e : R + —>• M + is an open-loop control function, that tends to zero as t goes to + 00 . 

Let us briefly describe our approach. Following a similar device as in 0, setting fi = i, and introducing 
the new unknown function ?/(•) = x (•) + fiv (•), we can equivalently rewrite (|3aD - (13bp as 

x(t) = prox /j4 , 

y (t) + {y (f)) + fie ( t ) prox^ (y (t)) = 0 

where prox^ is the proximal mapping associated to /i<F. Recall that prox^ = (/ + fid&) 1 is the resolvent 
of index fi > 0 of the maximal monotone operator 9d>, and = ^(/ — (I + yd^) 1 ) is its Yosida 

approximation of index fi > 0. As a key point of our analysis, we notice that prox $ is a gradient vector 
field, namely prox^ = V"0, with 

(4) i>(y) = m($*)a 02/^ , 

where <f>* is the Fenchel conjugate of <f>. Doing so, we can reformulate our dynamic in the form 

(5a) x(t) = (I + fid^y 1 (y (t )), 

(5b) y ( t ) + {y (t)) + e (t) V flip {y (f)) = 0. 

Equation (I5bl) is a particular case of the multiscale dynamic 

(6) y (t) + d@(y(t)) + e (t) (y (t)) 9 0 

where 0 and T are two convex potential functions. Following [3], T will be referred to as the "viscosity 
function". A detailed study of the asymptotic behavior of the orbits of © can be found in 0, DU, D3], m, 
[18] . Following 0 and [T4] , we focus our attention on the case where the parametrization f i-> e (f) satisfies 
the following "slow" decay property 

r+00 

/ e ( t ) dt = + 00 . 

Jo 

This condition expresses that e (•) does not tend to zero too rapidly, which allows the term e (•) x (•) to be 
effective asymptotically. In that case, we will show an asymptotic selection property. Precisely, in Theorem 
13.21 under some additional moderate growth property on e(-), we will show that, for any trajectory (x,v) of 
(I3all - (l3b() . x(-) converges weakly to the minimizer of $ which also minimizes ip over all minima of $. Then 
we show that this element is nothing but the element of minimal norm of the solution set argmin$, i.e., 

x(t) ->■ proj argmin$ 0 as t —> + 00 . 

Thus we recover the classical Tikhonov viscosity selection principle, which consists in selecting the solution 
of minimal norm. 

This result can be viewed as an asymptotic selection property: by using such a slow control e, one can force 
all the trajectories to converge to the same equilibrium, which here is the equilibrium of minimal norm. This 
makes a sharp contrast with the non controlled situation, or fast control, where the limits of the trajectories 
depend on the initial data, and are in general difficult to identify. 

The paper is organized as follows: we first show the existence and uniqueness of a strong global solution 
to the Cauchy problem (l3al) - (l3bll . Then, we study the asymptotic convergence as t goes to +00 of the 
trajectories of (15al) - (l3bl) . In our main result, Theorem 13.21 under the key assumption that e (•) is a "slow 
control", i.e., / 0 + °° e (t) dt = + 00 , and has moderate growth, we show the weak convergence of the trajectories 
toward the optimal solution of problem (V) of minimum norm. When $ is a convex differentiable function 
whose gradient is Lipschitz continuous, we show that the convergence holds for the strong topology. Finally, 
we examine some variants of this principle of hierarchical minimization. 
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2. Existence and Uniqueness of Global Solutions 
We consider the Cauchy problem for the differential inclusion system (15al) - (l3bl) 

(7a) v (t) £ 9$ (x (t)) 

(7b) Xx ( t ) + v (t) + v (t) + £ ( t ) x (t) = 0 

(7c) x (0) = xo, v (0) = vq 

First, we are going to define a notion of strong solution to the above system. Then, we shall reformulate 
this system with the help of the Minty representation of 9$. Finally, we shall prove the existence and 
uniqueness of a strong solution to system flTal) (17cl) . by applying the Cauchy-Lipschitz theorem to this 
equivalent formulation. 


2.1. Definition of strong solutions. We say that the pair (x (•), w (•)) is a strong global solution of 
(ITall (l7cl) iff the following properties are satisfied: 

i) x (•), v (•) : [0, +oo[ —> TL are absolutely continuous on each interval [0, 6], 0 < b < +oo; 

ii ) v ( t ) £ 9$ (x (t)) for all t £ [0, +oo[; 

Hi) Xx (t) + i) (t) + v (t) + e (t) x (t) = 0 for almost all t £ [0, +oo[; 
iv ) x (0) = xo, v (0) = vq. 

2.2. Equivalent formulation as a classical differential equation. In order to solve system (pal) ([7c]) 

we use Minty’s device. Set 

1 

Let us rewrite inclusion (I7al) by using the following equivalences: for any t £ [0, +oo[ 

(8) v(t) £ 9<F(x(f)) <:=> 

(9) x (t) + nv ( t ) £ x (t) + /i9$ (x (t)) 

(10) x (t) = (I + p9<h) -1 (x (t) + (j,v (t)). 

Let us introduce the new unknown function y : [0, +oo[ —>■ 7~L which is defined for t £ [0, +oo[ by 

(11) y (t) := x (t) + nv (t ), 

and rewrite the system (I7al) (17^ with the help of (x,y). From (flOll and (fill) 

x(t) = (I + /i9d>) _1 (y(t)), 

v ( t ) = ~ ( y (*) “ ( 7 + M^)” 1 ( y (t))) • 

Equivalently, 

(12) x (f) = prox^ (y (t)) ; 

(13) v(t ) = V$ M (y(t)), 

where prox^^ is the proximal mapping associated to /r'F. Recall that prox^ $ = (I + /i9$) 1 is the resolvent 
of index p > 0 of the maximal monotone operator 9d>, and is its Yosida approximation of index /z > 0. 

Let us show how (I7bl) can be reformulated as a classical differential equation with respect to y (•). First, 
let us rewrite m as 

(14) x (t) + nv ( t ) + n v ( t) + )Ji£ ( t ) x (f) = 0. 

Differentiating (ED, and using (1141) we obtain 

(15) y(t) = x (t) + ni) (t) 

(16) = —/j,v (t) — fie (t) x (t). 

From ED, ED, an d El) we deduce that 

y (t) + (y (t)) + fie ( t) prox M$ (y (t)) = 0. 
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Finally, the (x, y ) system can be written as 

(17a) x (t) = prox M$ {y (t)) 

(17b) y (t) + /iV^ {y {t)) + ye (t) prox^ (y (t)) = 0. 

Conversely, if y (•) is a solution of (I17bl> . then {x (•), v (•)) with x ( t ) = prox^ (y (7)) , v (t) = (y (7)) 
is a solution of (17a|) - (17c]l . Let us stress the fact that the operators prox ;jfI , : 77 —> 77, : 77 —>• 77 are 

everywhere defined and Lipschitz continuous, which makes this system relevant to the Cauchy-Lipschitz 
theorem. 


2.3. Global existence and uniqueness results. Let us state our main result of existence and uniqueness 
for the system (17al) (ITcl) . 

Theorem 2.1. Suppose that <f> : 77 —> M U {+oo} is a convex lower semicontinuous proper function, 
and A > 0 is a positive constant. Let e : R+ —i M+ be a nonnegative locally integrable function, and 
(a;o,uo) E 77 x 77 be such that Vo E 9$ (xo). Then the following properties hold: 

i) there exists a unique strong global solution (x(-), u(-)) : [0,+oo[ — > 77 x 77 of the Cauchy problem 
iTTali - ifTcl) .- 

ii) the solution pair (x (•), v (•)) of li 7all - 1 7c |) can be represented as follows: for any t E [0, +oo[, 

(18) x{t) = prox^(y{t)); 

(19) u(t) = V<My(t)), 

where y (•) : [0, +oo[ — > 77 is the unique strong global solution of the Cauchy problem 
(20a) y (t) + {y (t)) + ye (t) prox ^ {y (t)) = 0, 

(20b) y(0) = x o +yv o . 


Proof. Let us first prove the existence and uniqueness of a strong global solution of the Cauchy problem 
(I20al) - (l20bl) . The Cauchy problem (I20al) (I20bl) can be equivalently written in abstract form, as the following 
non-autonomous differential system 


( 21 ) 

with 

( 22 ) 

(23) 

(24) 


y{t) = F(t,y(t )); 
y (0) = x 0 + yv o, 


F{t,y) = G{t,y) + K(t,y), 
G(t,y) = -/rV$ M (y), 

K {t, y) = (t) prox^^, (y). 


In order to apply the Cauchy Lipschitz theorem to (1211) , let us first examine the Lipschitz continuity prop¬ 
erties of F (7, •). 

(a) Take arbitrary yi E 77, i = 1,2. The Yosida approximation is —Lipschitz continuous (see urn 
and hence, for any t > 0, G (t, •) : 77 —> 77 is nonexpansive, i.e., 


(25) 


\\G(t,y 2 ) ~G(t,y i)|| < \\y 2 - yi\\ ■ 


By the nonexpansive property of the resolvent operators we have 


(26) || I< ( t,y 2 ) - K (t,y i)|| < ye(t) || y 2 - yi\\ . 

Hence, 

(27) \\F (7, y 2 ) - F(7,yi)|| < (1 + ye{t)) \\y 2 - yi\\ ■ 

Since e (•) is nonegative and locally integrable, (EH) shows that the Lipschitz constant Lp{t ) = (1 + ye (7)) 
of F(t,-) satisfies 

(28) Lf (•) E L 1 ([0, 6]) for any 0 < b < +oo. 

(b) Let us show that 

(29) 


Vy E 77, V6 > 0, F(-,y) E L 1 ([0,6];77). 
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Returning to the definition (1221) of F, we deduce that 

\\F(t,y)\\ < y || (y)|| + ye (t) ||pro X/1$ {y)\\ . 

By assumption, e : R+ —> K + is a nonnegative locally integrable function, which gives (1251) . From (1271) and 
(129[) . by Cauchy-Lipschitz theorem (see [17], [21] for the nonautonomous version used here), we deduce the 
existence and uniqueness of a strong global solution of the Cauchy problem (1211) . and hence of (120al) - (j20bf) . 

(2) Let us return to the initial problem (|7a|) - (f7c|) . Given y (•) : [0, +oo[ —> H which is the unique strong 
solution of Cauchy problem (120aD - (120bl) . let us define x (•), v (•) : [0, +oo[ —> H by 

(30) x (t) = prox M$ (y (t )), v (t) = (y (t)). 

(a) Let us show that x (•) , u(-) are absolutely continuous on each bounded interval, and satisfy (17al) - (17cl) . 
Let us give arbitrary y\ G "H, y 2 G 'H- By the nonexpansive property of the resolvents, we have 

(31) ||prox M$ (y 2 ) - prox^ (yi)|| < ||y 2 - 2/i|| ■ 

Assuming that s,t G [0,6], by taking j/i = y(s), y 2 = y(t) in (1211) . and owing to the definition of the 
absolute continuity property, we deduce that x (t) = prox^ (y (t)) is absolutely continuous on [0, b] for any 
b > 0. As a linear combination of two absolutely continuous functions, the same property holds true for 
v{t) = A (y (t) -x(t)). 

Moreover, for any t € [0, +oo] 

v (t) e 6$ (x (t )), y(t)=x (t) + yv (t). 

Differentiation of the above equation shows that, for almost every t > 0, 

x (t) + yv (t) = y{t). 

On the other hand, owing to v ( t ) = (y (t)), x (t) = prox M$ (y (t)), (120al) can be equivalently written as 

y ( t ) + yv ( t ) + ye ( t ) x (t) = 0. 

Combining the two above equations, we obtain 

x (t) + yv (t) + yv (t) + ye (t) x (t) = 0. 

From y = j, we conclude that (x (•), v (•)) is a solution of system C5J-CZ0- 
Regarding the initial condition, we observe that 

(32) y (0) = x 0 + yv o 

(33) = x (0) + yv (0), 
with vq 6 6$ (xo) and v (0) G 6$ (x (0)). Hence 

x (0) = x 0 = (/ + yd^) -1 (x 0 + yv 0 ). 

After simplification, we obtain v (0) = vq- 

(b) Let us now prove the uniqueness. Suppose that 

x (•), v (•) : [0, +oo[ —■» H 

is a solution pair of (|7a|) - ([7c|) . Defining y = j and 

(34) y(t) = x (t) + yv (t) 

we conclude that y (•) is absolutely continuous, yo = xo + yv o, and for any t G [0, +oo[ 

(35) x (t) = (I + yd^Y 1 {y (t )), v (t) = (y (t)). 

Since the functions involved in the definition (1341) of y (•), namely x (•) and v (•), are differentiable for almost 
all t G [0, +oo[, we have for almost t G [0, +oo[ 

y(t) = x (t) + yv (t) 

= -y (v (t) + v (t) + e (t) x (t)) + yv ( t ). 

Since v ( t ) = (y (t)), we finally obtain 

V (t) + (y (t)) + ye (t) prox M$ (y (t)) = 0. 
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Moreover 


2 /o = x 0 + nv 0 ■ 

Arguing as before, by the Cauchy-Lipschitz theorem, the solution y (•) of the above system is uniquely 
determined, and locally absolutely continuous. Thus, by (1351) . x (•) and v (•) are uniquely determined. □ 


3. Asymptotic analysis and convergence properties 

In this section, we study the asymptotic behavior, as t —> +oo, of the trajectories of system tfZD-CZb]). 
Let us recall our standing assumption, namely the parametrization e (•) is supposed to be nonnegative, and 
locally integrable. In view of the asymptotic analysis, we also suppose that e (t) —> 0 as t —>• oo, and satisfies 
the "slow" decay property 

+oo 

£ (t) dt = Too. 

By Theorem I2TI for any given Cauchy data Vq £ <9$ (xo), the above properties guarantee the existence 
and uniqueness of a global solution of system ([7a D - ([7bl) -([7c |l . From now on in this section, (x(-) , u(-)) : 
[0, Too[ —> H x % is the solution of (17a1)-(17bl)-(17cl). We first study the asymptotic behavior, as t —> Too, of 
the trajectories of the associated system (HHD 

y (t) T n'(y (t)) T pe (t) prox ^ {y (t)) = 0 

whose existence is guaranteed by Theorem l2.ll The central point of our analysis is to reformulate this system 
as a multi-scale gradient system, which will allow us to use the known results concerning the asymptotic 
behavior, and the hierarchical selection property for such systems. 

3.1. Preliminary results. Let us state some definitions and classical properties that will be useful (see [5], 
cd, m, m for an extended presentation of these notions): 

Definition 3.1. Let / and g be functions from T-L to MU {Too}. The infimal convolution (or epi-sum ) of / 
and g is the function fOg : H —> [—oo, Too] which is defined by 

fUg(x) = inf (/ (0 T g (x - 0) ■ 

Definition 3.2. Let / : Ti. —» M U {Too}, 7 £ R++- The Moreau envelope of / of parameter 7 is defined by 

f ^ fD (>i 2 ). 

Definition 3.3. Let / :H-lRU {+ 00 } be a convex lower semicontinuous proper function, and let x £ H. 
Then prox^x is the unique point in % that satisfies 

/i(x) = min (7 (0 T ||x — £|| 2 
The operator proxy : T~L —> TL is called the proximity operator, or proximal mapping of /. 

Definition 3.4. Let / : T~i —> R U {Too}. The conjugate (or Legendre-Fenchel transform, or Fenchel 
conjugate) of / is 

/* :K->RU {Too} : u 1 —> sup ((x, u) — f (x)). 

x£H 

Remark 3.1. Let f : H —>M. U {Too} be proper then 

/* (0) = — inf /■ 

rt 

f is lower semi continuous and convex if and only if 

/ = /**• 

Remark 3.2. Let / and g be functions from H to R U {Too}. Then 

(fDg)* = f*+g*. 

Conversely, if one of the functions (/ or g) is continuous at a point of the domain of the other, then 

(f + g)* = ra g *. 


)=/( p: 


\ 1 II II2 

roxjx) T — 11x — prox^x|| . 
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Remark 3.3. a) Let {+ 00 } be proper, and 7 G R++. Set / = <p + ||-|| 2 . Then Vm € TL 

/* («) = \ IM | 2 ~¥> 7 ( 7 u )- 

b) Let C be a nonempty subset of TL, and let f = Sc + ||-| /2, where dc is the indicator function of the set 
C (Sc (x) = 0 for x £ TL, +00 outwards). Then 

r = (n-n 2 -4) a 

where d c is the distance function to the set C. For the proof, set <p = Sc and 7 = 1 in Remark 13.31 

In the next lemma, we show that the proximal mapping can be written as the gradient of a convex 
differentiable function. This result will play a crucial role in our analysis. 

Lemma 3.1. Let <I> : TL —> R U {+ 00 } be a proper convex lower semicontinuous function, and let p > 0. 
Then, the proximal mapping prox : TL —> Tt can be written as the gradient 

prox^ = S7ip 

of the convex continuously differentiable function if : TL —> ffi. which is defined, for any y £ TL, by 

if(y) =/i($*)jL 

where the Fenchel conjugate of' f>. 


Proof. For any y £ TL, set 

x = prox M$ (y) . 

By definition of the proximal mapping, we have the following equivalent formulations 

x= (I + y9<F ) _1 (y) 
y £ (I + pd<5>) (x) 
y — x £ pd<& (x) 

- (y — x) £ 9$ (x). 


From (9$) = 9<f>* and the above equality, we successively obtain 

1 


x £ 9<h* (y — x)j ; 

—y £ — (y — x) + — 9d>* ( — (y — x) ) = ( I + —9$ 


T 


— (y — x) = I I H—9$* 

p 


T 

-1 


T 


x = p 




y — [ I H—9$’ 


-V 

T 

1 


x = p(j- prox±$„) (p^) ' 


-(y-x)); 


By the definition of the Yosida approximation of index ^ > 0 of the maximal monotone operator 


x = V 




-y 




By the classical derivation chain rule, we deduce that the the proximal mapping prox^y : TL —>■ TL is the 
gradient of the convex continuously differentiable function ip : TL —> R which is defined, for any y £ TL, by 

1 


ip(y) = p(<5>*) i ( -y ) ■ 


□ 
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Let us further analyze the function ip, and give equivalent formulations which come with different proofs 
of the above lemma. By Definition 13.21 of the Moreau envelope of $ of parameter /i 


(36) 


3v(y) = inf { $ 0*0 + 77- \\y - x 




2y 


and since the Yosida approximation of the subdifferential of $ is the Frechet derivative of Moreau envelope, 

( y ) = -( y - J %* y ) 
y 

(y) = y~ J?y. 

By definition, we have prox^ (y) = J^y = (I + /i(9<f>) _1 (y). Hence 

AtV<fv {y) = y- prox M$ (y). 

Prox^ (y) = y- /zV$ M (y) 

) (y)- 


- V D 

Let us make the link with the previous formulation of the prox as a gradient, and show that, for any y G Ti. 


By we have 


^11 yf-v^viy) = 


i 


i 


llyll -y^(y) = \-y ; 


— p inf l $ (x) H- ||y — x 


16 H 


ig n 


2 /i 

1 


- y inf { ®{x) + — \\y\\ - (y,x) + 


ig n 


y sup 




2y 

1 

2y 

V,x) - ( $(x) + 




2y 


sup ^ (y, x) — y ( <f> (x) + — ||x" 2 


2 /i 


By using Remark 13.21 concerning the conjugate of a sum, we obtain 


1 


\\y\\ -v®n(y) = A* $ + 


= y 


(Vn 


2 /i 

M II ||2 




= 


-y 


So we obtain the same function ip as given in Lemma |3.1I Note that by Remark 13.31 the equivalent (dual) 
formulation of ip given by ip(y) = \ ||j/|| — /x<f» M (y), which is written as a d.c. function, is actually a convex 
function. 

3.2. Asymptotic hierarchical minimization. Let us study the asymptotic behavior of the trajectories 
of system CIMZ9- We consider the equivalent system (120al) . which, by Lemma 13.11 can be formulated as 
follows: 


(37) 

(38) 

where, for any y £ 77 

(39) 

(40) 


x (t) = prox M$ (y (t)) 
y (t) + V0 {y (t)) + e (t) V>F {y (t)) = 0, 


Q(y) := /r$ M (y); 
*(y) = y 2 ($*)i 
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Note that 0 and ^ are two convex continuously differentiable functions. We are within the framework of 
the multiscale gradient system (MAG) e , with a positive control t >-»• eft) that converges to 0 as t —>• oo, 

(MAG) e yft) + d@(y(t)) + e(t)dAf(y(t)) 9 0 , 

that has been considered by Attouch-Czarnecki in [6] . Let us recall this general abstract result, that we 
formulate with notations adapted to our setting. Since 0 enters (MAG) e only by its subdifferential, it is 
not a restrictive assumption to assume this potential to be nonnegative, with its infimal value equal to zero 
(substracting the infimal value does not affect the subdifferential). 


Theorem 3.1. (Attouch-Czarnecki, [6],) Let 

• 0 : TL —> R + U {+oo} be a closed convex proper function, such that C = argmin© = © _1 (0) ^ 0. 

• : LL —> R U {+oo} be a closed convex proper function, such that S = argmin{\1/|argmin©} 0. 
Let us assume that, 


r TOO 

{'Hi) s Vp G R(Nc), / Q*(e(t)p) - ac(e(t)p)dt < +oo. 

Jo 

r +OO 

( 0.2) e £(•) is a non increasing function of class C 1 , such that limt^+oo eft) = 0, e(t)dt = +00, 

Jo 


and for some k > 0, —fee 2 < e. 

Let y(-) be a strong solution of (MAG) e . Then: 


(f) 

weak convergence 

zli/oo £ S = argmin{\k|argmin©}, 

(**) 

minimizing properties 

lim Q(y(t)) = 0; 

£—>■+00 

Jj‘+0^(1/+) = min + argm j ne ; 

(Hi) 


Vz € S lim II y(t) — z\\ exists ; 
£—>• + 00 

(iv) 

estimations 

lim -7770(j/(*)) = 0; 
t-j+oo eft) 



r+00 



/ Q(y(t))dt < +00; 

Jo 


w - lim y(t) = y 

£—>•+00 


lim sup 

T-t-\-00 


e(t) (*(y(t )) 


min if 


argnnne 


dt < + 00 . 


By specializing this result to our setting, we will obtain the weak convergence of y(-) to a particular 
minimizer of <f>, which is the solution of a hierarchical minimization property. The convergence of x(-) is less 
immediate, and will follow from an energetical argument. 


Analysis of the condition (Hi) e : The condition 


(Ri) e Vp G R(N C ), 


+00 


Q*(e(t)p) - ac(e(t)p)dt < + 00 , 


plays a crucial role in our asymptotic analysis. Before proceeding in the discussion of this hypothesis, we 
recall some classical notions from convex analysis, that will be useful. 

• ac is the support function of C, 

a c (x*) = sup (x*,c). 
cec 

• Nc (x ) is the normal cone to C at x , 

N C (x) = {z* € Tl : (x*, c — x) < 0 for all c £ C} if x £ C, and 0 otherwise. 


• R (Nc) is the range of Nc, i.e. p £ R (Nc) if and only if p G Nc (x) for some x £ C. 

• Note that Sf, = ac where Sc is the indicator function of C, 


Se¬ 



if xGC 
otherwise. 
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Observe that in (Hi) , all the terms in the integral are nonnegative. Indeed, since 0 is bounded from 
above by the indicator function of the set C, i.e. 0 < Sq (recall that 0 = 0 on C), the reverse inequality 
holds for their Fenchel conjugates, whence 

0* (e (t) p) — ac (e (t) p) > 0 Vp G TL. 

Thus, Hypothesis (TLi) t means that, for all p G R(Nc ) the nonnegative function 

t eA [ 0 * (e (t) p) - a c (e ft) p)] 

is integrable on (0, +oo). For more clarity, let us discuss the following special case: Suppose that 

0 (a) > ^dis 2 (x, C ), 

for some r > 0. Then 0* (x) < ^ ||x || 2 + ac (x) and 


0 * (z)-ac(z) < ^ ||z|| 2 . 

Hence, in this situation (TLi) e is satisfied if the following condition on e(-) is satisfied: 

+oo 

e 2 ft) < Too. 

In this situation, the moderate growth condition on e(-), can be formulated as 

e(-) G L 2 ( 0, Too) \ L 1 f 0, Too). 

Let us return to the general situation, and summarize our results in the following theorem, which is our main 
statement. 


Theorem 3.2. Let <f> : TL —> K + U{Too} be a closed convex function, such that C = argmind’ = <f> 1 (0) ^ 0. 
Let us assume that, 

r+°° 1 1 

• (Hi) e Vp G R(N C ), / p$*(-e(t)p) - a c (e(t)p) T -\\e(t)p\\ 2 dt < Too; 

J 0 M ^ 

• (TL 2 ) £ e(-) is a nonincreasing function of class C 1 , Lipschitz continuous on [0, Too[, and such that 

r+00 

lim i _ ) ._|_ 00 eft) = 0, / e(t)dt = Too, and for some k > 0, —he 2 < e. 

Jo 

Then, for any trajectory (x (•) ,v (•)) : [0, Too[ —5- TL x TL solution of A7a\) - F7b\) . with y (t) = x (t) T pv (t) 

( i) weak convergence w — limt-^+oo y(t) = pi'°jargmin$ 0 - 
Let us further assume that $(0) < Too. Then 

(■ ii ) weak convergence w — lim x(t) = w — lim y(t) = proj ars -min 4 >^> 

(in) strong convergence s — lim v(t) = 0 , and hence s — lim x(t) — y(t) = 0 . 

£—>■+00 £—>-+oo 

Proof. Let us apply Theorem 13. II with 0(y) = p^ ll (y), and ^ (y) = p 2 (<f>*)j_ which are two convex 

continuously differentiable functions. By the general properties of the Moreau enveloppe, the function 0 is 
still nonnegative, and 


argmin© = argmind*^ = argmind> = C. 
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Moreover, on C we have 0 = 0. Let us particularize the conditions and {W. 2 ) e to our setting. Let us 

first compute 0*, the conjugate of 0. For any z £ U 

Q*(z) = 


Hence conditions ('Hi) and (^ 2 ) £ of Theorem 13. 1 1 are satisfied. As a consequence, we obtain the convergence 
of y(-) to a solution y 00 of the constrained minimization problem 

(41) min{'F(;r) : i£C). 

Let us write the first-order optimality condition satisfied by y a0 . We have 

(42) V4/(?/ 00 ) + Nc(yoo) 9 0. 

Since = /iprox^, equivalently 

(43) MProx^j/oo) + Nc{yoo) 9 0. 

Noticing that yoo £ C, and that z = prox^z for z £ C = argmin<f>, we obtain 

(44) /xyoo + N c (yoo) 9 0. 

By definition of Nq , equivalently, the following property is satisfied 

(0 — j/oo) c — j/oo) < 0 VceC. 

Since j/oo £ C, this is the condition of the obtuse angle that characterizes the projection of the origin on C. 
Thus 

(45) yoo = proj c (0). 

We have obtained that y(-) converges weakly to the element of minimal norm of the solution set C, that’s 
item (i). In order to pass from the convergence of y(-) to the convergence of x(-) we use the relation (fT21) 

x (t) = prox^ (y (t)) 

that links the two variables. 

In a finite dimensional setting, we can conclude the strong convergence of x(-) thanks to the continuity of the 
proximal mapping, and using again the fact that the set C of minimizers of $ is invariant by the proximal 
mapping prox^, i.e., 

prox M$ (y) = y for all y £ C = argmin'L. 

In an infinite dimensional setting, we are going to use the particular structure of our dynamical system, and 
an energetical argument to show that 

(46) x(t) — y(t) —> 0 strongly as t —> + 00 . 

This will result from the finite energy property 

n OO 

(47) / \\y(t)\\ 2 dt < + 00 . 

Jo 

To obtain (l47l) , take the scalar product of equation (1381) with y (t). We obtain 

(48) ||* (t) || 2 + J t (B (y m + e (t) | (* (y m = 0 . 


sup {(z,y) - ixQ^y)} 


M SU P < (~z,y) - 4> M (y) 


V 




//$* 




m 11 1 112 
77 \\-z 
2 y 


-z I + ~\\ z \\ 
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After integration by parts we obtain, for any T > 0 

(49) / T || y (t) \\ 2 dt + 0 (y (T)) -Q(y (0)) + e (T) * (y (T)) - e (0) * (y (0)) - T <+)4> (y (t)) dt = 0. 

Jo Jo 

By assumption $(0) < +oo. By Remark l3.ll <£** = <f>, we equivalently have inf <f>* > — oo, and hence 

inf 4/ = inf y 2 ($* + = y 2 inf <f>* > — oo. 

A* 

Set m := inf 'F. From (|49|) , and e(t) < 0 (recall that e(-) is a nonincreasing function) we deduce that 
r T r T 

(50) 


[ II V {t) || 2 dt <&{y (0)) + e(0)T (y (0)) + |m|e (T) + m [ e(t)dt. 
Jo Jo 


Since the above majorization is valid for any T > 0, and e is bounded (it decreases to zero), we obtain (1471) . 
We now observe that y(-) is Lipschitz continuous on [0, +oo[. This follows from equation (1381) . and the 
following argument. Since y(-) is converging weakly, it is bounded. By the Lipschitz continuity of the 
operators V© = /iV^ and V’F = /rprox^ (which are therefore bounded on bounded sets), and by equation 
(1381) . we deduce that y(-) is bounded, and hence y(-) is Lipschitz continuous on [0,+oo[. 

Using again equation (1551) . and the Lipschitz continuity properties of V0 and VT, y, and e (for this last 
property note that for some c > 0, < — ke 2 < e < 0), we deduce that t > y(t) is Lipschitz continuous on 

[0, +oo[. Hence y(-) belongs to L 2 ([0, +oo[; Ji ), and is Lipschitz continuous. By a classical result this implies 

lim y(t) = 0 . 

t —>- + oo 

Returning to (l38l) . and noticing that e(<)V4'(y(t)) —>• 0, we obtain 

lim V0(y(t)) = 0. 

£->-+oo 

Since V0(y(t)) = y(t) — prox M$ (y(t)) = y(t) — x(t), we finally obtain that x(t) — y(t) converges strongly to 
zero as t — > +oo, which clearly implies that x(- ) and y(-) converge weakly to the same limit, which is the 
solution of a hierarchical minimization problem. □ 

Example: Let us return to our model situation where 


$ (x) > 2 dis (+ c ) ■ 

for some r > 0. Then (x) < ^ ||x|| 2 + ac (x) and 

(z) -a c {z) < F || 2 :|| 2 . 

After elementary computation, one can verify that, in this situation, (7U) e is satisfied if the following 
condition on e(-) is satisfied: 

r*+oo 


r 

Jo 


(t) < +oo. 


Thus, in this situation, the moderate growth condition on e(-), can be formulated as 

e(-) G L 2 ( 0, +oo) \ L 1 (0, +oo). 

3.3. Strong convergence. Let us now examine the strong convergence properties of the trajectories. Let 
us first consider the variable y(-). Following [6j Theorem 2.2], and equation (1551) . the strong convergence of 
y(-) will result from the strong monotonicity property of V4/ = /iprox^. We recall that is said to be 
strongly monotone if there exists some a > 0 such that for any x € domV'F, and y £ domV'F 

(V’F (x) — V\t r (y) ,x — y) > a ||x — y\\ 2 . 

This property turns out to be equivalent to a regularity property for <f>, as stated in the following Lemma. 

Lemma 3.2. Let $ be a convex differentiable function whose gradient is L-Lipschitz continuous for some 
L > 0. Then, for any y > 0 such that yL < 1, the proximal mapping prox $ is strongly monotone. 
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Proof. Take y, : , i = 1,2. By definition of prox /i 4 > (y i ), prox M$ (?/,) + pV<f>(prox /i$ (y i )) = y t . Taking the 
difference of the two equations, and multiplying scalarly by y 2 — yi, we obtain 

(prox Al 4 > (y 2 ) -prox M$ (?/i),i /2 - yi) + P (V$(prox M<I ,(?/ 2 ) - V$(prox M$ (yi), y 2 - yf) = || y 2 - yi\\ 2 ■ 

Then use the Cauchy-Schwarz inequality, the L-Lipschitz continuity of V3>, and the fact that the proximal 
mapping is nonexpansive to obtain 

(prox M$ (y 2 ) -prox M$ (?/i ),y 2 - yi ) > (1 - pL)\\y 2 - yi\\ 2 ■ 

Conversely, one can easily establish that the strong monotonicity of the proximal mapping implies that $ is 
a convex differentiable function whose gradient is Lipschitz continuous. □ 

We can now complete Theorem 13.21 as follows. 

Theorem 3.3. Let us make the assumptions of Theorem Id.‘A and assume moreover that $ is a convex 
differentiable function whose gradient is L-Lipschitz continuous for some L > 0. Then for pL < 1, we have 
the strong convergence property of x(-) and y(-) to the element of minimal norm of C = argmin<f> 0. 

Proof By Theorem 13.21 item (in), x(t) — y(t ) converges strongly to zero as t —► + 00 . Hence we just 
need to prove that y(-) converges strongly . Since pL < 1 , by Lemma [3.21 the operator V\k = pprox^ 
is strongly monotone. Thus we are in the situation examined in [51 Theorem 2.2], which gives the strong 
convergence property. Another equivalent approach consists in noticing that by Theorem 13.11 we have 
w - lim f _j . +00 y(t) = proj ar g m ; M ,0 and ^(y(t)) -3 T(proj ar g rn ; n<l) 0). From this we easily deduce that the 
strong convexity of \k implies the strong convergence of y(-). We recover our result by noticing that the strong 
convexity of \k is equivalent to the strong monotonicity of its gradient, i.e., of the proximal mapping. □ 


4. Other viscosity selection principles 


Let us now examine the more general situation 
(51a) v (t) £ <9<h (x (t)) 

(51b) Xx ( t ) + i) (t) + v (t) + e ( t) dg(x (t)) 3 0, 

where g is a convex viscosity function. 

Using the Minty transform, this system can be equivalently written as 


(52a) x(t) = prox M$ (y (t)) 

(52b) y (t) + (y (t)) + pe (t) dg( prox^ (y (t))) 3 0. 

In order to recover exactly the Tikhonov approximation, we look for some g such that, for all y £ LI 


Equivalently 
We obtain 


d(pg)( prox M$ (y)) = y. 
(I + pd®)- 1 = (d(pg))- 1 . 
I + pd<5> = d(pg), 


that is, for all y £ TL 

T9(y) = l\\y\\ 2 + p®(y)- 

Thus, by taking g(y) = ^j||y|| 2 + $(j/), and 0 (y) = pQ^ (y), equation (I52bl) can be equivalently written as 
(53) y (t) + V0 (y (t)) + e ( t ) y ( t) = 0. 

Equation (1551) is a particular case of the (SDC) system (steepest descent with control) 


(SDC) y ( t ) + dO (y (t)) +e(t)y ( t ) = 0. 

Concerning the case f 0 °°e (t) dt = + 00 , the first general convergence result was in [Ti|] (based on previous 
work by m ), and also requires e (•) to be nonincreasing, and converges to zero for t —> + 00 . Under these 
conditions, each trajectory of (1531) converges strongly to the point of minimal norm in C = argmin© = 
argmin<I>. In [15], it is proved that the convergence result still holds without assuming e(-) to be nonin¬ 
creasing. 
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When g(£) = 2^||£|| 2 + < h(£), the dynamical system (151al) - (l51bl) becomes 
(54a) v (t) e (x (t)) 

(54b) Xx (t ) + v (t) + v (t) + e (t) a; (t) + u (t)^j = 0. 

Equivalently 

(55a) v (t) € d§ (x (t)) 

(55b) x (t) + fiv (t) + nil + £ (t))v (t) + £ (t) x (t) = 0. 

As in the preceding section, by application of the Cauchy-Lipschitz theorem (recall that 0 is differentiable, 
and its gradient is Lipschitz continuous), we can show that (1551) admits a strong global solution y (•). Then, 
(a:(-) ,!>(•)) with x {■) = prox^ (?/(•)) and v (•) = {y (■)) is a strong solution of (I55all - (l55bl) . 

Let us summarize our result in the following theorem. 


Theorem 4.1. Let <f> : Tl — > RU{+oo} be a closed, convex, proper function, such that C = argmin<f> ^ 0. 
Let us assume that 

(i) lim e (t) = 0, 

t —)- + oo 

(ii ) / Q +00 e (t) dt = +oo. 

Then, for any trajectory (x (•) ,v (•)) : [0, +oo[ —► TLxTL solution of L r >5a\) ~ \55b\) . with y (t) = x (t)+fj,v (t) 
the following strong convergence propery holds: 

s — lim x (t) = s — lim y (t) = Proj 0. 

t H-OO t H-OO argmin$ 


Proof. We are in the situation examined in |15l Theorem 2], which gives the strong convergence of each 
trajectory y (•) of (1531) towards the point of minimal norm in C = argmin4>. In order to pass from the 
convergence of y(-) to the convergence of x(-) we use the relation (I52al) 


x ( t ) = prox^ (y (t)) 

that links the two variables. From the continuity property of the proximal mapping for the strong topology 
(indeed, it is a nonexpansive mapping), we deduce that each trajectory x{j of (I55a|l - (I55bl) converges strongly 

to prox^ I Proj 0 ) = Proj 0. To obtain this last equality, we use the fact that the set C of minimizers 

\arg min 4> J arg min 

of $ is invariant by the proximal mapping prox^, i.e., 

prox M$ (?/) = y for all y £ C = argmin4>. 

□ 


5. Perspective 

Let us list some interesting questions to be examined in the future: 

(1) Examine the discrete, algorithmical version, and the corresponding asymptotic selection property 
for the forward-backward algorithm. 

( 2 ) Study the case where X(t) depends on t in an open-loop form, as in [9]. 

(3) Study the case where the Levenberg-Marquart regularization term is given in a closed-loop form, 
A (t) = a(||a;(t)||) as in 0- 

(4) Examine these questions for the related dynamical systems which have been considered in p]. 
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